Monocular SLAM Supported Object Recognition

In this work, we develop a monocular SLAM-aware object recognition system that is able to achieve considerably stronger recognition performance, as compared to classical object recognition systems that function on a frame-by-frame basis. By incorporating several key ideas including multi-view object proposals and efficient feature encoding methods, our proposed system is able to detect and robustly recognize objects in its environment using a single RGB camera in near-constant time. Through experiments, we illustrate the utility of using such a system to effectively detect and recognize objects, incorporating multiple object viewpoint detections into a unified prediction hypothesis. The performance of the proposed recognition system is evaluated on the UW RGB-D Dataset, showing strong recognition performance and scalable run-time performance compared to current state-of-the-art recognition systems.Comment: Accepted to appear at Robotics: Science and Systems 2015, Rome, Ital

Towards Visual Ego-motion Learning in Robots

Many model-based Visual Odometry (VO) algorithms have been proposed in the past decade, often restricted to the type of camera optics, or the underlying motion manifold observed. We envision robots to be able to learn and perform these tasks, in a minimally supervised setting, as they gain more experience. To this end, we propose a fully trainable solution to visual ego-motion estimation for varied camera optics. We propose a visual ego-motion learning architecture that maps observed optical flow vectors to an ego-motion density estimate via a Mixture Density Network (MDN). By modeling the architecture as a Conditional Variational Autoencoder (C-VAE), our model is able to provide introspective reasoning and prediction for ego-motion induced scene-flow. Additionally, our proposed model is especially amenable to bootstrapped ego-motion learning in robots where the supervision in ego-motion estimation for a particular camera sensor can be obtained from standard navigation-based sensor fusion strategies (GPS/INS and wheel-odometry fusion). Through experiments, we show the utility of our proposed approach in enabling the concept of self-supervised learning for visual ego-motion estimation in autonomous robots.Comment: Conference paper; Submitted to IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2017, Vancouver CA; 8 pages, 8 figures, 2 table

High-Performance and Tunable Stereo Reconstruction

Traditional stereo algorithms have focused their efforts on reconstruction quality and have largely avoided prioritizing for run time performance. Robots, on the other hand, require quick maneuverability and effective computation to observe its immediate environment and perform tasks within it. In this work, we propose a high-performance and tunable stereo disparity estimation method, with a peak frame-rate of 120Hz (VGA resolution, on a single CPU-thread), that can potentially enable robots to quickly reconstruct their immediate surroundings and maneuver at high-speeds. Our key contribution is a disparity estimation algorithm that iteratively approximates the scene depth via a piece-wise planar mesh from stereo imagery, with a fast depth validation step for semi-dense reconstruction. The mesh is initially seeded with sparsely matched keypoints, and is recursively tessellated and refined as needed (via a resampling stage), to provide the desired stereo disparity accuracy. The inherent simplicity and speed of our approach, with the ability to tune it to a desired reconstruction quality and runtime performance makes it a compelling solution for applications in high-speed vehicles.Comment: Accepted to International Conference on Robotics and Automation (ICRA) 2016; 8 pages, 5 figure

Learning Articulated Motions From Visual Demonstration

Many functional elements of human homes and workplaces consist of rigid components which are connected through one or more sliding or rotating linkages. Examples include doors and drawers of cabinets and appliances; laptops; and swivel office chairs. A robotic mobile manipulator would benefit from the ability to acquire kinematic models of such objects from observation. This paper describes a method by which a robot can acquire an object model by capturing depth imagery of the object as a human moves it through its range of motion. We envision that in future, a machine newly introduced to an environment could be shown by its human user the articulated objects particular to that environment, inferring from these "visual demonstrations" enough information to actuate each object independently of the user. Our method employs sparse (markerless) feature tracking, motion segmentation, component pose estimation, and articulation learning; it does not require prior object models. Using the method, a robot can observe an object being exercised, infer a kinematic model incorporating rigid, prismatic and revolute joints, then use the model to predict the object's motion from a novel vantage point. We evaluate the method's performance, and compare it to that of a previously published technique, for a variety of household objects.Comment: Published in Robotics: Science and Systems X, Berkeley, CA. ISBN: 978-0-9923747-0-

Computer-Controlled Machines for Pathology Slide Sorting and Cataloguing System

Final report for Team 11 of ME450, Fall 2008 semesterThe objective of these projects is to develop an opto-mechatronic system with optical scanning of bar codes and sample shapes as well as servomotors to automate the sorting and cataloguing of a collection of pathology glass slides and paraffin blocks. These machines are related to the reduction of human error using automation in healthcare. The UM hospital generates thousand of slides and hundreds of blocks per day, which must be correctly catalogued and filed away for future retrieval and reference. This manual process is very tedious and prone to errors due to its long duration and repetitive nature. Once a slide or block is incorrectly catalogued, it is virtually impossible to locate it again. Patients may need to provide additional tissue samples or errors may result with specimens from other patients. To realize this automation an embedded computer control and monitoring system will be used to drive a series of servo motors to move slides through the mechanical system, while sensors provide feedback to monitor its progression. Each slide is marked with a 2D-barcode, which must be scanned to determine the correct path through the system. Slides may have the incorrect orientation when entering the system for the barcode to be read and thus the orientation must be automatically corrected by the system. Once the slides have been catalogued, they must be ejected from the machine in a specified order into a specially designed receptacle compatible with a robotic storage system. The embedded system must utilize a connection to a network distributed SQL (Structured Query Language) database to record relevant slide and block information. Two machines, one for slides and another for blocks, have different requirements because the physical size and weight difference of slides and blocks. (This project is the slide sorting machine)Peter Lucas (Anatomic Pathology, U of M Medical School) and Ulysses Balis (Pathology Informatics, U of M Medical School)http://deepblue.lib.umich.edu/bitstream/2027.42/61920/1/ME450 Fall2008 Final Report - Team 11 - Slide Sorter.pd